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IMAGE PROCESSING METHOD, APPARATUS AND STORAGE MEDIUM 

FIELD OF THE INVENTION 

5 The present invention relates to image processing, and 

particularly to the method and apparatus for processing image 
that includes candidates for human face region, and storage 
medium for storing computer programs used in processing image 
that includes candidates for human face region. 

10 

BACKGROUND OF THE INVENTION 

A number of techniques are known for detecting areas 
of interest in an image, such as a face or other identified 

15 object of interest . Face detection is an area of particular 
interest, as face recognition has importance not only for 
image processing, but also for identification and security 
purposes, and for human- computer interface purposes. A 
human- computer interface not only identifies the location 

20 of a face, if a face is present, it may also identify the 
particular face, and may understand facial expressions and 
gestures . 

Many studies on automatic face detection have been 
reported recently. References for example include "Face 
25 Detection and Rotations Estimation using Color 

Information, " the 5th IEEE International Workshop on Robot 
and Human Communication, 1996, pp 341-346, and "Face 
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Detection from Color Images Using a Fuzzy Pattern Matching 
Method, " IEEE Transaction on Pattern Analysis and Machine 
Intelligence, vol. 21, no. 6, June 1999. 

All the conventional methods of detecting human faces 
5 have their own advantages as well as shortcomings depending 
upon different algorithms used for processing images . Some 
methods are accurate but are complex and time-consuming. 

Further, it is also very important to detect whether 
an image is a portrait . An image containing any human faces 

10 is called a portrait. In recent years, in association with 
the rapid spread of modern IT devices, a request for 
accurately determining a portrait has been increasing. For 
example, if the image can be judged as a portrait, the 
printer can adjust its parameters optimized for the 

15 portrait when it prints the image. Then the image's 
printing quality can be greatly improved. For this reason, 
there has been a strong demand for a device or a process, 
which can easily and automatically detect a portrait in 
an image . 

20 Importantly, none of the conventional methods of 

detecting human faces saves the detecting result to the 
image, which makes it not convenient for further processing 
of the image if a particular process is needed for human 
face regions . 

25 It is thus needed in the art to develop a method and 

apparatus that can identify human face regions within the 
image or determine whether an image is a portrait and save 
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the result of the identifying process or the result of the 
determining process for further processing of the image. 

SUMMARY OF THE INVENTION 

5 

The first objective of the invention is to provide 
a method and apparatus of processing an image that will 
save information of candidates for human face region to 
the image for processing of the image at a later time. 
10 The second objective of the invention is to provide 

a method and apparatus of processing an image in which 
information of candidates for human face region has been 
saved . 

The third objective of the invention is to provide 
15 a method and apparatus of processing an image that will 
save information of portrait to the image for processing 
of the image at a later time. 

The fourth objective of the invention is to provide 
a method and apparatus of processing an image in which 
20 information of portrait has been saved. 

For the attainment of the above objectives, the 
present invention provides a method of processing an image , 
characterized by comprising steps of: 

identifying one candidate for human face region 
25 within said image; 

calculating a probability that said candidate for 
human face region represents a human face; and 
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saving said probability as attached information to 
said image. 

The present invention also provides a method of 
processing an image, characterized by comprising steps of : 
5 identifying one candidate for human face region 

within said image; 

calculating a probability that said candidate for 
human face region represents a human face; 

judging whether or not said candidate for human face 
10 region represents a human face by comparing said 
probability with a threshold; and 

saving a result of said step of judging as attached 
information to said image. 

The present invention further provides an apparatus 
15 for processing an image, characterized by comprising: 

a candidate selector for identifying one candidate 
for human face region within said image; 

a probability calculator for calculating a 
probability that said candidate for human face region 
20 represents a human face; and 

a probability writer for writing said probability as 
attached information to said image. 

The present invention further provides an apparatus 
for processing an image, characterized by comprising: 
25 a candidate selector for identifying one candidate 

for human face region within said image; 

a probability calculator for calculating a 
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probability that said candidate for human face region 
represents a human face; and 

a judging unit for judging whether said candidate for 
human face region represents a human face by comparing said 
5 probability with a threshold; and 

a judging result writer for writing an output of said 
judging unit as attached information to said image. 

The present invention further provides a method of 
processing an image, characterized by comprising steps of : 
10 obtaining at least one probability, each of said at 

least one probability representing possibility that one 
candidate for human face region within said image 
represents a human face; 

calculating a portrait probability for said image 
15 based on said at least one probability; and 

saving said portrait probability as attached 
information to said image. 

The present invention further provides a method of 
processing an image, characterized by comprising steps of : 
20 obtaining at least one probability, each of said at 

least one probability representing possibility that one 
candidate for human face region within said image 
represents a human face; 

calculating a portrait probability for said image 
25 based on said at least one probability; and 

judging whether or not said image is a portrait by 
comparing said portrait probability with a threshold; and 
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saving a result of said step of judging as attached 
information to said image. 

The present invention further provides an apparatus 
for processing an image, characterized by comprising: 
5 a probability for candidate for human face region 

calculator, for obtaining at least one probability, each 
of said at least one probability representing possibility 
that one candidate for human face region within said image 
represents a human face; 
10 a portrait probability calculator for calculating a 

portrait probability of said image that said image is 
portrait; and 

a probability writer for writing said probability as 
attached information to said image. 
15 The present invention further provides an apparatus 

for processing an image, characterized by comprising: 

a probability for candidate for human face region 
calculator, for obtaining at least one probability, each 
of said at least one probability representing possibility 
20 that one candidate for human face region within said image 
represents a human face; 

a portrait probability calculator for calculating a 
probability that said candidate for human face region 
represents a human face; and 
25 a judging unit for judging whether said image is 

portrait by comparing said probability with a threshold; 
and 
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a judging result writer for writing an output of said 
judging unit as attached information to said image. 

The present invention further provides a storage 
medium storing a computer program for processing an image, 
5 characterized by comprising: 

code means for identifying one candidate " for human 
face region within said image; 

code means for calculating a probability that said 
candidate for human face region represents a human face; 
10 and 

code means for saving said probability as attached 
information to said image. 

The present invention further provides a storage 
medium storing a computer program for processing an image, 
15 characterized by comprising: 

code means for identifying one candidate for human 
face region within said image; 

code means for calculating a probability that said 
candidate for human face region represents a human face; 
20 code means for judging whether or not said candidate 

for human face region represents a human face by comparing 
said probability with a threshold; and 

code means for saving a result of said step of judging 
as attached information to said image. 
25 The present invention further provides a storage 

medium storing a computer program for processing an image, 
characterized by comprising: 
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code means for obtaining at least one probability, 
each of said at least one probability representing 
possibility that one candidate for human face region within 
said image represents a human face; 
5 code means for calculating a portrait probability for 

said image based on said at least one probability; and 

code means for saving said portrait probability as 
attached information to said image. 

The present invention further provides a storage 
10 medium storing a computer program for processing an image, 
characterized by comprising: 

code means for obtaining at least one probability, 
each of said at least one probability representing 
possibility that one candidate for human face region within 
15 said image represents a human face; 

code means for calculating a portrait probability for 
said image based on said at least one probability; and 

code means for judging whether or not said image is 
a portrait by comparing said portrait probability with a 
20 threshold; and 

code means for saving a result of said step of judging 
as attached information to said image. 

According to the method of the present invention, 
results of identifying candidates for human face regions 
25 or results of calculating portrait probability will be 
saved to the image, and further processes to be conducted 
on the image will be facilitated. 
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Also, a conventional image processing apparatus can 
be trained to possess the ability of detecting human faces . 
The method of detecting human face according to the present 
invention is both accurate and quick. 
5 Additionally, the method of the present invention can 

be easily combined with different algorithms used in 
conventional methods of detecting human faces so as to fit 
in different situations. 

According to the method of the present invention, it 
10 can be judged whether the image is a portrait or not and 
the result of the judgment is saved to the image. When the 
image is judged as a portrait, the printer can adjust its 
parameters optimized for the portrait such that the 
printing quality of the image will be improved. Further, 
15 some image processing methods optimized for portrait can 
be performed in display devices on the image so as to 
improve the display quality of the image. 

Other features and advantages of the present 
invention should be apparent from the following 
20 description of the preferred embodiments, taken in 
conjunction with the accompanying drawings, which 
illustrate, by way of example, the principles of the 
invention . 

25 BRIEF DESCRIPTION OF THE DRAWGINS 

Fig. 1 is the flowchart of the method of training an 
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image processing apparatus in one embodiment of the present 
invention; 

Fig. 2 is the flowchart of the method of processing 
an image according to the present invention, in which an 
5 image processing apparatus that has been trained with the 
method shown in Fig. 1 is used; 

Fig . 3 is the flowchart of another method of processing 
an image according to the present invention, in which a 
plurality of image processing apparatuses that have been 
10 trained with the method shown in Fig. 1 are used; 

Fig . 4 is the flowchart of another method of processing 
an image to which probability for at least one candidate 
for human face region has been saved; 

Fig. 5 is the flowchart of a method of identifying 
15 a person in an image to which probability for at least one 
candidate for human face region has been saved; 

Fig. 6 is a schematic block diagram of an apparatus 
for processing an image according to the present invention; 
Fig. 7 is a schematic block diagram of another 
20 apparatus for processing an image according to the present 
invention; 

Fig. 8 is a schematic block diagram of an apparatus 
for processing an image to which probability for at least 
one candidate for human face region has been saved; 
25 Fig. 9 shows a training sequence including 1000 

training samples , i.e., image regions Al , A2 , A3, A100 0; 
Fig. 10 shows two image regions Bl, B2 for testing; 
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Fig. 11 is the flowchart of another method of 
processing an image according to the present invention, 
in which a portrait probability is calculated for the 
image ; 

5 Fig. 12 is the flowchart of another method of 

processing an image to which portrait probability has been 
saved; 

Fig. 13 is a schematic block diagram of another 
apparatus for processing an image according to the present 
10 invention; 

Fig. 14 is a schematic block diagram of another 
apparatus for processing an image according to the present 
invention; 

Fig. 15 is a schematic block diagram of another 
15 apparatus for processing an image to which portrait 
probability has been saved; 

Fig. 16 shows another image for which a portrait 
probability is calculated; 

Fig. 17 schematically shows an image processing 
20 system in which each method shown in Figs. 1 to 5, 11, and 
12 can be implemented; 

Fig. 18 shows an example of the apparatus for 
processing an image. 

25 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

The embodiments of the present invention will be 
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described in detail. In the following description, as to 
how to identify one candidate for human face region within 
an image, reference can be made to Chinese Patent 
Application No. 00127067.2, which was filed by the same 
5 applicant on September 15,- 2000, and made public on April 
10, 2002. This application is incorporated here for 
reference. However, the method of identifying candidates 
for human face regions disclosed in Chinese Patent 
Application No. 00127067.2 constitutes no restriction to 

10 the present invention. Any conventional method of 
identifying candidates for human face regions within an 
image may be utilized in the present invention. 

Fig. 1 is the flowchart of the method of training an 
image processing apparatus in one embodiment of the present 

15 invention. The process starts at step 101. At step 102, 
a plurality of image regions are inputted. These image 
regions may be obtained from a single image or a plurality 
of images. Some of these image regions represent genuine 
human faces. It is known beforehand which image regions 

20 represent genuine human faces. These image regions are 
called as "training samples." In Fig. 1, the number of 
training samples are N, wherein N is an integer greater 
than one . 

At step 103, for each of the image regions inputted 
25 in step 102, a predetermined algorithm will be applied 
thereon such that a M- dimensional vector will be generated, 
wherein M is an integer equal to or greater, than one. In 
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the case that M takes the value of one, the predetermined 
algorithm generates a scalar value for each inputted image 
region . 

In the above way, a plurality of M-dimensional vectors 
5 will be generated. The number of the generated 
M-dimensional vectors is the same as N. Since it is known 
beforehand which training sample (i.e., image region) 
represents a genuine human face , it will also be known which 
M-dimensional vector corresponds to a genuine human face. 

10 The present invention does not concern the detailed 

process within the predetermined algorithm as long as the 
algorithm generates a M-dimensional vector for each 
inputted image region. Thus, the predetermined algorithm 
can be any conventional method of processing data for an 

15 image . The vector generated by the predetermined algorithm 
represents some characteristics of the image region upon 
which the predetermined algorithm is applied. Two examples 
of these algorithms will be given later with reference to 
Fig. 9 (see Example One, and Example Two) . 

20 After step 103, N M-dimensional vectors will be 

generated, and they are distributed in a M-dimensional 
space . 

Following steps 104 to 108 constitute a way of dividing 
the M-dimensional space into a plurality of subspaces, the 

M 

25 number of which is denoted as I~[Kj and within which the 

j=l 

same number of M-dimensional vectors are distributed. The 
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number of the M-dimensional vectors distributed in each 

M 

subspace is denoted as N/]^[Kj , wherein Kl , K2 , KM are 

j=l 

integers greater than one. 

It should be note that there are numerous ways of 

M 

5 dividing the M-dimensional space into [ | Kj subspaces. 

Steps 104 to 108 just show an example and do not constitute 
a restriction to the present invention. 

In step 104, the value "one" is assigned to variable 

i . 

10 In step 105, all the M-dimensional vectors 

distributed in each subspace will be aligned with the i-th 
axis by the values of the i-th components of the 
M-dimensional vectors to be aligned. 

In step 106, the i-th axis in each subspace will be 

15 divided into Ki intervals, such that the M-dimensional 

i 

space will be correspondingly divided into [~[Kj subspaces, 

j=l 

and the number of the M-dimensional vectors distributed 

i 

in each subspace will be N/]~jKj . 

In step 107, variable i increases by one. 
20 In step 108, itisj udged whether variable i is greater 

than M. If the judging result of step 108 is negative, the 
process will go to step 105; otherwise go to step 109. 
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In step 109, the probability for each subspace is 
calculated. Within one subspace, the number of the 
M-dimensional vectors corresponding to genuine human 
faces is first counted. Then, the total number of the 
5 M-dimensional vectors distributed in the subspace, i.e., 
M 

N/J~[Kj, is divided by the above number of M-dimensional 
j=l 

vectors corresponding to genuine human faces. The result 

of the division is taken as the probability for the subspace. 

The probability for a subspace means the possibility that 
10 a vector distributed in this subspace corresponds to a 

genuine human face. 

In optional step 110, the positions and probabilities 

for all the subspaces are saved for example to a memory 

or storage within the image processing apparatus. 
15 In step 111, the training process ends. 

To facilitate the understanding of the flowchart 

shown in Fig. 1, two examples are given as follows. 

Example One 

20 

Refer to Fig. 9, which shows a training sequence 
composed of 1000 training samples , i.e., image regions Al , 
A2, A3, A1000. Thus, N in Fig. 1 takes the value of 1000. 

In Fig. 9, it is known beforehand that which image 
25 regions represent genuine human faces and which image 
regions do not. For instance, image regions Al , A5 
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represent genuine human faces, and image regions A2 , A3, 
A4 , A6 do not represent human faces. 

The predetermined algorithm applied in Example One 
generates a scalar value, i.e., a M-dimensional vector 
5 where M=l. Being an example, the predetermined algorithm 
used in this example generates the proportion of the areas 
belonging to skin color to the whole image region. 

Take image region Al for example. The total number 
of pixels in image region Al is 10000, wherein the number 
10 of pixels belonging to skin color is 8000. Thus, the 
proportion of areas belonging to skin color to the whole 
image region is 8000/10000 = 0.8. 

When respectively applying the above predetermined 
algorithm to image regions Al , A2 , A1000, 1000 scalar 

15 values, called training scalar values, will be obtained, 
which are shown as : 

0.8, 0.2, 0.3, 0.5, 0.7, 0. 1... . 

Next, along the real axis, aligning the above training 
scalar values in an ascending order, and obtaining the 
20 sequence : 

... , 0.1, ... , 0.2, ... , 0.3, ... , 0.5, ... , 0.7, ... , 0.8, 

Then, dividing the real axis into M intervals, such 
that each interval will include the same number of training 
scalar values. The number of training scalar values within 
25 each interval equals to N/M . 

Assuming M=10, the real axis will be divided into 10 
intervals (i.e., 10 subspaces, each of which is 
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1 -dimensional) , which for example are: 
(-00, 0.11] , 
(0.11, 0.2] , 
(0.2, 0.32], 
5 (0.32,0 .39] , 

(0.39, 0.45] , 
(0.45, 0.56] , 
(0.56, 0.66] , 
(0.66, 0 . 73] , 
10 (0 . 73 , 0 . 85] , 

(0.85, +00) . 

The intervals are open at left ends and closed at right 
ends, or closed at left ends and open at right ends. In 
each interval, i.e., 1 -dimensional subspace, there are 
15 N/M=1000/10=100 training scalar values. 

Next, calculating the probability for each interval. 
For the 10 intervals divided as above, the numbers of 
training scalar values corresponding to genuine human 
faces within the 10 intervals are: 
20 5, 11, 16, 28 ,32 ,44, 52, 61, 77,43. 

The total number of training scalar values in each 
interval is N/M = 1000/10 = 100. 

Then, the probability for each of the 10 intervals 
is respectively: 
25 0.05, 0.11, 0.16, 0.28, 0.32, 0.44, 0.52, 0.61, 0.77, 

0 .43 . 

In the last step, the positions and probabilities are 
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saved for the 10 intervals. 

Example Two 

5 Fig. 9 shows a training sequence including 1000 

training samples , i.e., image regions Al, A2 , A3, A1000. 
In this example, only image regions Al , A2 , A3, ... , A900 
are used. Thus, N in Fig. 1 takes the value of 900. 

As stated above, it is known beforehand that which 
10 image regions represent genuine human faces and which image 
regions do not. For instance, image regions Al , A5 
represent genuine human faces, and image regions A2 , A3, 
A4 , A6 do not represent human faces. 

The predetermined algorithm applied in Example Two 
15 generates a 2 -dimensional vector, i.e., a M-dimensional 
vector where M=2 . Being an example, the predetermined 
algorithm used in this example generates the average and 
weighted angle between the gradients of the gray level 
distribution and a reference distribution in an annular 
20 region. For detailed explanation of this algorithm, please 
refer to Chinese Patent Application No. 01132807. X. 

When respectively applying this algorithm to image 
regions Al , A2 , ... , A900, the following 2 -Dimensional 
vectors called as training vectors will be obtained: 
25 (0.23, 0.14), (-0.6, -0.71), (0.44, 0.51), (0.52, 

0.74), (-0.16, -0.22), (0.58, 0.46), ... 

Next, on the 1st axis, i.e., the real axis, aligning 
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the 900 2 -Dimensional vectors by the values of the 1st 
components of these vectors in an ascending order, 
obtaining the following sequence: 

(-0.6,-0.71),..., (-0.16,-0.22),..., ( 0 . 23 , 0 . 14 ) , 
5 (0.44, 0.51), (0.52, 0.74), (0.58,0.46), ... 

Then, dividing the real axis into P intervals, and 
correspondingly the 2 -Dimensional space into P subspaces, 
such that each of the P subspaces will include N/P 
2 -Dimensional vectors . 
10 Assuming P=10, the 10 intervals will be: 

(-oo, -0.6] , 
(-0.6, -0.33] , 
(-0.33, -0.12] , 
(-0.12, 0.09] , 
15 (0.09,0. 15] , 

(0.15, 0.26] , 
(0.26, 0.44] , 
(0.44, 0 . 57] , 
(0.57, 0.73] , 
20 (0.73, +oo) . 

All the intervals are open at left ends and closed 
at right ends, or closed at left ends and open at right 
ends . 

Within each subspace, there are N/P = 90 training 
25 vectors . 

Next, within each subspace, aligning the training 
vectors along the 2nd axis by the values of the 2nd 
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components of the training vectors in an ascending order. 

For instance, in the subspace corresponding to 
interval (-0.12,0.09] , the following training vectors are 
distributed : 

5 (-0.1, 0.2), (-0.05, 0.01), (-0.03, 0.3), 

(0.01, -0.1), (0.03, -0.22), (-0.06, -0.5), ... 

Aligning these vectors by the values of the 2nd 
components results in the following sequence: 

(-0 . 06 , -0 . 5) , (0 . 03, -0 .22) , ( 0 . 01 , - 0 . 1 ) , ... , 
10 (-0.05, 0.01), (-0.1, 0.2), (-0.03, 0.3), ... 

Within each subspace, dividing the 2nd axis into Q 
intervals, and correspondingly each subspace into Q 
subspaces, such that each subspace eventually obtained 
includes the same number of 2 -Dimensional vectors, the 
15 number of which is N/ (P*Q) . 

Assuming Q=9, for each subspace obtained by dividing 
the real axis, the 2nd axis is divided into 9 intervals. 

Take the subspace corresponding to interval 
(-0.12,0.09] for example. The resulted 9 intervals are: 
20 (-oo # -0.5] , 

(-0.5, -0.35] , 
(-0.35, -0.18] , 
(-0.18, 0.04] , 
(0 . 04 , 0 . 17] , 
25 (0.17, 0.31] , 

(0.31, 0.54] , 
(0.54, 0.77] , 
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(0 . 77, +00) . 

All the intervals are open at left ends and closed 
at right ends, or closed at left ends and open at right 
ends . 

5 In each subspace, there are N/ (P*Q) = 10 training 

vectors . 

In the above way, the 2 -Dimensional space is 
eventually divided into the following (P*Q)=90 subspaces : 
((-co, -0.6), (-co, -0.53)), ((-co, -0.6], (0.71, 

10 +co) ) , 

((-0.6, -0.33), (-co, -0.58)), ... , ((-0.6, -0.33], 
(0.56, +00)), 

((-0.12, 0.09), (-00, -0.5)), ((-0.12, 0.09], (0.04, 

15 0.17)), ((-0.12, 0.09), (0.77, +00)) 

( (0.73, +00) , (-co, -0.65) ) , ( (0.73, +00) , (0.61, +co) ) 
In each subspace, N/ (P*Q) = 10 training vectors are 
distributed . 

20 In the next step, calculating the probability for each 

subspace . 

Assuming the number of training vectors corresponding 
to genuine human faces are respectively: 

1 / ••• / 2 , 0 , ... , 3 , ... , 3 , ... , 8 , ... , 2 , ... , 0 , ... , 1 
25 Since the total number of training vectors 

distributed in each subspace is N/ (P*Q) = 900/ (10*9) = 10, 
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the probabilities for the 90 subspaces are: 

0.1/ ... , 0.2, 0 , ... , 0.3/ ... , 0.3/ ... , 0.8/ ... / 0*2/ ... / 0 / ••• / 

0.1. 

In the last step, the positions and probabilities are 
5 saved for the 90 subspaces. 

Fig. 2 is the flowchart of the method of processing 
an image according to the present invention, in which an 
image processing apparatus that has been trained with the 
method shown in Fig. 1 is used. The process begins at step 
10 201. An image is inputted at step 202. In order to detect 
a human face within the image to be processed, one candidate 
for human face region is identified within the image at 
step 203. At step 204, the data for the above candidate 
for human face region is inputted to an image processing 
15 apparatus that has been trained with the method shown in 
Fig. 1. 

At step 205, within the image processing apparatus, 
the predetermined algorithm that has been used in the 
training process of the image processing apparatus is 
20 applied to the data for the candidate for human face region, 
and a M-dimensional vector is generated for the candidate 
for human face region. 

At step 206, one subspace, in which the above 

M-dimensional vector is located, is identified among the 

M M 
25 ]~ J Kj subspaces. These J~[Kj subspaces are formed during 

the training process of the image processing apparatus, 
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and their information (for instance, the positions, and 
probabilities) has been saved in the image processing 
apparatus . 

At step 207, the value of the probability for the 
5 identified subspace is assigned to the candidate for human 
face region (identified at step 203) . 

In this way, the probability for each candidate for 
human face region can be easily obtained within an image 
processing apparatus that has been trained with the method 

M 

10 shown in Fig. 1. Also, since the probabilities for f~jKj 

j=l 

subspaces have been saved in the image processing apparatus , 
the calculation amount can be greatly reduced in the 
detection of a human face. 

It should be understood that steps 204 to 207 just 

15 constitute an embodiment of the present invention and make 
no restriction to the invention. Any conventional methods 
may be adopted if a probability that the candidate for human 
face region, as identified in step 203, represents a human 
face can be calculated by these conventional methods. 

20 At step 2 08, it is judged whether the candidate for 

human face region represents a human face or not by 
comparing the probability for the candidate for human face 
region with a threshold. 

At step 209, the result of the judgment is saved as 

25 attached information to the image, for example, to the 
header file or footer file of the image in a predetermined 
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format. At step 209, identification information of the 
candidate for human face region can also be saved as 
additional attached information to the image, for example, 
to the header file or footer file of the image in a 
5 predetermined format . 

At step 210, the probability for the candidate for 
human face region is saved as attached information to the 
image, for example, to the header file or footer file of 
the image in a predetermined format. At step 210, 

10 identification information of the candidate for human face 
region can also be saved as additional attached information 
to the image, for example, to the header file or footer 
file of the image in a predetermined format. 

In both of steps 209 and 210, the predetermined format 

15 for saving the attached information and additional 
attached information is not important and does not 
constitute a restriction to the present invention. Any 
conventional format or data structure for saving data may 
be used. 

20 The image to which judging results or probabilities 

have been saved can be used widely. Figs. 4 and 5 will 
illustrate some application of such kind of images. 
The above process ends at step 211. 

25 Example Three 

Now refer to Fig. 10, which shows two image regions 
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Bl, B2 for testing. As shown in Fig. 10, image region Bl 
shows a human face, while image region B2 does not. The 
following description will demonstrate the excellent 
results of the detecting method of the present invention. 
5 Take the algorithm used in Example One for example. 

If image region Bl is identified as one candidate for 
human face region, a scalar value 0.75 will be generated 
by the algorithm, which falls in the interval (0.73, 0 . 85] . 
Since the probability for this interval is 0.77, the 

10 probability of image region Bl also takes the value of 0.77. 

If image region B2 is identified as one candidate for 
human face region, a scalar value 0.31 will be generated 
by the algorithm, which falls in the interval (0.2, 0.32). 
Since the probability of this interval is 0.16, the 

15 probability of image region B2 also takes the value of 0.16. 

Apparently, the probability for one candidate for 
human face region which actually represents a human face 
is increased (from 0.75 to 0.77) , and the probability for 
one candidate for human face region which actually does 

20 not represent a human face is decreased (from 0 . 31 to 0 . 16) . 
That is , the accuracy of detecting a human face is increased 
in the present invention. 

Example Four 

25 

Reference is also made to Fig. 10 which shows two image 
regions Bl, B2 for testing. 
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Take the algorithm used in Example Two for example. 

If image region Bl is identified as one candidate for 
human face region, a 2 -dimensional vector (0.05, 0.11) will 
be generated by the algorithm, which falls in the subspace 
5 ((-0.12, 0.09], (0.04, 0.17]). Since the probability of 
this subspace is 0.8, the probability of image region Bl 
also takes the value of 0.8. 

If image region B2 is identified as one candidate for 
human face region, a 2 -dimensional vector (-0.71, -0.66) 
10 will be generated by the algorithm, which falls in the 
subspace ((-oo, -0.6), (-oo, -0.53)). Since the probability 
of this subspace is 0.1, the probability of image region 
B2 also takes the value of 0 . 1 . 

A different algorithm is used in this example, and 
15 the accuracy of detecting a human face is further increased, 
compared with Example Three. 

Fig. 3 is the flowchart of another method of processing 
an image according to the present invention, in which a 
plurality of image processing apparatuses that have been 
20 trained with the method shown in Fig. 1 are used. 

The process starts at step 301. Then at step 302, the 
data for an image is inputted. At step 303, one candidate 
for human face region is identified within the inputted 
image . 

25 At steps 304 through 306 , a plurality of probabilities , 

which are called intermediate probabilities, for the 
candidate for human face region are obtained using a 
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plurality of image processing apparatuses that have been 
trained with the method shown in Fig. 1. The number of the 
plurality of image processing apparatuses for example is 
K. K is an integer equal to or greater than one . The detailed 
5 process of obtaining a probability using an individual 
image processing apparatus resembles the method shown in 
Fig. 2. 

Different algorithms may be used in different image 
processing apparatuses. But, of course, for each image 
10 processing apparatus, the algorithm used in the training 
process of the image processing apparatus and the algorithm 
used in the detailed process of obtaining the probability 
should be the same. 

After steps 304 through 306, K intermediate 
15 probabilities pi, p2 , pK are obtained. 

At step 307, a probability is calculated for the 
candidate for human face region based on the above 
intermediate probabilities pi, p2 , pK, with the 

following equation : 

20 P=«a-no-p0) 

i=i 

wherein a is a factor that is less than but very closed 
to one . 

It should be understood that steps 304 to 307 just 
constitute an embodiment of the present invention and make 
25 no restriction to the invention. Any conventional methods 
may be adopted if a probability that the candidate for human 



27 



Canon Ref: CFP2513US King & Wood Ref: EIE031501US 

face region, as identified in step 303, represents a human 
face can be calculated by these conventional methods. 

At step 308, it is judged whether the candidate for 
human face region represents a human face or not by 
5 comparing the probability for the candidate for human face 
region with a threshold. 

At step 3 09, the result of the judgment is saved as 
attached information to the image, for example, to the 
header file or footer file of the image in a predetermined 

10 format. At step 309, identification information of the 
candidate for human face region can also be saved as 
additional attached information to the image, for example, 
to the header file or footer file of the image in a 
predetermined format . 

15 At step 310, the probability for the candidate for 

human face region is saved as attached information to the 
image, for example, to the header file or footer file of 
the image in a predetermined format. At step 310, 
identification information of the candidate for human face 

20 region can also be saved as additional attached information 
to the image, for example, to the header file or footer 
file of the image in a predetermined format. 

In both of steps 309 and 310, the predetermined format 
for saving the attached information and additional 

25 attached information is not important and does not 
constitute a restriction to the present invention. Any 
conventional format or data structure for saving data may 
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be used. 

The image to which judging results or probabilities 
have been saved can be used widely. Figs. 4 and 5 will 
illustrate some applications of such kind of images. 
5 The above process ends at step 311. 

Example Five 

Reference is also made to Fig. 10 which shows two image 
10 regions Bl, B2 for testing. 

As stated above in Example Three and Example Four, 
the probabilities (i.e., intermediate probabilities) for 
image region Bl is 0.77 and 0.8. 

Let a be 0.9. 

15 The probability for image region Bl will be calculated 

as 0 . 9* (1- (1-0 . 77) * (1-0 . 8) ) = 0.86. 

As stated above in Example Three and Example Four, 
the probabilities (i.e. , intermediate probabilities) for 
image region B2 is 0.16 and 0.1. 
20 Let a be 0.9. 

The probability for image region B2 will be calculated 
as 0 . 9* (1- (1-0 . 16) * (1-0 . 1) ) = 0.22. 

From Fig. 3 and its description, it is apparent that 
if both K and a take the value of one, the method shown in 
25 Fig. 3 will be the same as that shown in Fig. 2. 

Fig . 4 is the flowchart of another method of processing 
an image to which probability for at least one candidate 
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for human face region has been saved. The process starts 
at step 401. At step 402, an image to which probability 
for at least one candidate for human face region has been 
saved is received. 
5 As stated above, the probability information might 

be saved as attached information in the header file or 
footer file of the image in a predetermined format. 

At step 403, the probability for one candidate for 
human face region is retrieved from the image, for example 

10 from the header file or footer file of the image. 

At step 404, the retrieved probability is compared 
with a threshold. At step 405, it is decided based on the 
comparison result of step 404 whether the current candidate 
for human face region represents a human face. The above 

15 threshold can be selected in such a way that any candidate 
for human face region having a probability above the 
threshold represents a human face. 

If the judging result of step 405 is positive, i.e., 
the candidate for human face region represents a human face, 

20 the process goes to step 406; otherwise, step 407. 

At step 406, a unique processing method optimized for ' 
human faces is performed in the candidate for human face 
region. The unique processing method might be performed 
by a printer, for example, a printer driver or an 

25 application program for printer 1713 in Fig. 17, such that 
human faces will be printed with improved printing quality . 
The unique processing method may also be performed by a 
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display, for example an application program for display 
1714 in Fig. 17, such that human faces will be displayed 
with high quality. 

The unique processing method in step 4 06 may be any 
5 process performed on an image. For example, it may be a 
process of correcting red eyes in an image, or a process 
of toning down skin color in an image. The value of the 
threshold used in step 404 may be selected as a higher value 
or as a lower value according to different applications. 

10 In the circumstance of red eye correcting, for example, 

it would be unacceptable to users if a portion of an image 
(for instance, a red logo) that is not an eye is corrected 
as a red eye. Thus, the threshold in step 4 04 could be 
set to a higher value so that less candidates for human 

15 face regions would be judged as representing human face 
and thus less portions of the image would be corrected as 
red eyes. 

In the circumstance of toning down human skin in an 
image, for example, it would be acceptable (sometimes 

20 unnoticeable) to users if a portion of an image (such as 
a table) that includes no human skin color is toned down. 
Thus, the threshold in step 4 04 could be set to a lower 
value so that more candidates for human face regions would 
be judged as representing human face and thus more portions 

25 of the image would be toned down. 

In one word, the threshold used in step 404 may be 
selected according to the users' satisfaction of the 
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processing result of the image. 

At step 407, a normal processing method is performed 
in the candidate for human face region. 

At step 408, other parts of the image that contain 
5 no candidates for human face region are processed. If there 
are other candidates for human face regions and their 
probabilities are included in the image, the process will 
go to step 403 . 

The process ends at step 409. 
10 Fig. 5 is the flowchart of a method of identifying 

a person in an image to which probability for at least one 
candidate for human face region has been saved. The process 
starts at step 501. At step 502, an image to which 
probability for at least one candidate for human face 
15 region has been saved is received. As stated above, the 
probability information might be saved as attached 
information in the header file or footer file of the image 
in a predetermined format. At step 503, the probability 
for one candidate for human face region is retrieved from 
20 the image, for example from the header file or footer file 
of the image . 

At step 504, the retrieved probability is compared 
with a threshold. At step 505, it is decided based on the 
comparison result of step 504 whether the current candidate 
25 for human face region represents a human face. The above 
threshold can be selected in such a way that any candidate 
for human face region having a probability above the 
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threshold represents a human face. 

The threshold used in step 504 can be selected in the 
same principle as in step 404 of Fig. 4. 

If the judging result of step 505 is positive, i.e., 
5 the candidate for human face region represents a human face, 
the process goes to step 506; otherwise, step 507. 

At step 506, a person is identified only based on the 
candidate for human face region, while at step 507 , a person 
is identified based on the whole image normally. It is easy 
10 to understand that if a person is identified only based 
on his/her face instead of the whole image, the 
identification process for this person will be greatly 
accelerated, and accuracy will be increased. 
The process ends at step 508. 
15 Fig. 6 is a schematic block diagram of an apparatus 

for processing an image according to the present invention. 
601 denotes an image inputting unit, 602 a candidate 
selector, 603 a vector generator, 604 a probability 
selector, 605 a probability memory, 606 a probability 
20 writer, 607 an image outputting unit. The key components 
of the apparatus shown in this figure are vector generator 
603, probability selector 604 and probability memory 605. 

As shown in Fig. 6, the components enclosed by the 
broken line constitute a probability calculator. Although 
25 it is shown that the probability calculator is composed 
of vector generator 603, probability selector 604 and 
probability memory 605, it should be understood that any 
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conventional components may be taken to form a probability 
calculator. That is, vector generator 603, probability 
selector 604 and probability memory 605 do not constitute 
a restriction to the probability calculator. The important 
5 is that the probability calculator will calculate a 
probability that one candidate for human face region 
represents a human face. 

The apparatus shown in Fig. 6 has been trained with 
the method shown in Fig. 1, and the positions and 

10 probabilities for all the subspaces are saved in 
probability memory 605. Probability memory 605 may take 
any form, such as ROM, EPROM, RAM, hard disk, etc. Different 
storage media and different storage schemes for positions 
and probabilities of the subspace do not constitute 

15 restrictions to the present invention. 

Image inputting unit 6 01 receives an image and inputs 
its data into the apparatus for processing. Candidate 
selector 602 selects a portion of the inputted image and 
identifies this portion as one candidate for human face 

20 region. Vector generator 603 performs the predetermined 
algorithm that has been used in the training process of 
the image processing apparatus on the data for the 
candidate for human face region, and generates a 
M-dimensional vector for the candidate for human face 

25 region. 

Since the algorithm used by vector generator 6 03 and 
the algorithm used in the training process of the image 



34 



Canon Ref: CFP2513US King & Wood Ref: EIE031501US 

processing apparatus are the same, the M- dimensional 
vector surely belong to a subspace whose position and 
probability has been saved in probability memory 605. 

Probability selector 604 retrieves a probability from 
5 probability memory 605 on the basis of the M-dimensional 
vector generated by vector generator 603. 

Probability writer 606 writes the probability 
retrieved by probability selector 604 as attached 
information into the image being processed, for instance, 
10 to its header file or footer file in a predetermined format . 
Probability writer 606 may also writes identification 
information of the candidate for human face region as 
additional attached information to the image, for example, 
to the header file or footer file of the image in a 
15 predetermined format. 

The predetermined format for saving the attached 
information and additional attached information is not 
important and does not constitute a restriction to the 
present invention. Any conventional format or data 
20 structure for saving data may be used. 

Image outputting unit 607 outputs the image for 
further processing. 

Fig. 7 is a schematic block diagram of another 
apparatus for processing an image according to the present 
25 invention. 701 denotes an image inputting unit, 702 a 
candidate selector, 703 a vector generator, 704 a 
probability selector, and 705 a probability memory. These 
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components function in the same way as corresponding 
components shown in Fig. 6. 

As shown in Fig. 7, the components enclosed by the 
broken line constitute a probability calculator. Although 
5 it is shown that the probability calculator is composed 
of vector generator 703, probability selector 704 and 
probability memory 705, it should be understood that any 
conventional components may be taken to form a probability 
calculator. That is, vector generator 703, probability 

10 selector 704 and probability memory 705 do not constitute 
a restriction to the probability calculator. The important 
is that the probability calculator will calculate a 
probability that one candidate for human face region 
represents a human face. 

15 7 06 denotes a judging unit , 707 a judging result writer, 

and 708 an image outputting unit. Judging unit 706 judges 
whether the candidate for human face region represents a 
human face or not by comparing the probability retrieved 
by probability selector 704 with a threshold. Judging 

20 result writer 707 writes the judging result outputted from 
judging unit 706 as attached information to the image, for 
example, to the header file or footer file of the image 
in a predetermined format. Judging result writer 707 may 
also writes identification information of the candidate 

25 for human face region as additional attached information 
to the image, for example, to the header file or footer 
file of the image in a predetermined format. 
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The threshold used in judging unit 607 can be selected 
in the same principle as in step 404 of Fig. 4. 

The predetermined format for saving the attached 
information and additional attached information is not 
5 important and does not constitute a restriction to the 
present invention. Any conventional format or data 
structure for saving data may be used. 

Image outputting unit 708 outputs the image for 
further processing . 
10 Fig. 8 is a schematic block diagram of an apparatus 

for processing an image to which probability for at least 
one candidate for human face region has been saved. 8 01 
denotes an image inputting unit, 802 a probability 
extractor, 8 03 a judging and control unit, 8 04 an image 
15 processing unit, 805 algorithm for human face, 806 
algorithm for normal image, 807 an image outputting unit. 

Image inputting unit 8 01 receives an image and inputs 
its data into the apparatus for processing. Within the 
image, probability for at least one candidate for human 
20 face region has been saved, for example, in step 210 of 
Fig. 2 or step 310 of Fig. 3, and by probability writer 
606 of Fig. 6. 

Probability extractor 802 retrieves from the image, 
for example from the header file or footer file of the image, 
25 the probability for one candidate for human face region. 
If identification information of the candidate for human 
face region has been saved in the image, probability 
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extractor 802 will also retrieves from the image the 
identification information of the candidate for human face 
region, that will be used by image processing unit 804. 

Judging and control unit 8 03 compares the retrieved 
5 probability with a threshold, decides based on the 
comparison result whether the current candidate for human 
face region represents a human face or not, and controls 
image processing unit 804 accordingly. 

The threshold used in judging and control unit 803 
10 can be selected in the same principle as in step 404 of 
Fig. 4. 

Image processing unit 804 processes the image 
inputted by image inputting unit 801 using different 
algorithms such as algorithm for human face 8 05 and 

15 algorithm for normal image 806 under the control of judging 
and control unit 803. If it is decided by judging and 
control unit 803 that the candidate for human face 
represents a human face, image processing unit 804 will 
process the candidate for human face that is identified 

20 by the identification information retrieved from the image 
with algorithm for human face 805 , otherwise with algorithm 
for normal image 806. Image processing unit 804 for 
instance is the component within a printer or a display 
for processing data to be printed or displayed, or a device 

25 for identifying an object or a person. 

Image outputting unit 8 07 outputs the image for 
further processing . 
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The following description refers to Figs. 11 to 16, 
which concerns portrait probability of an image. 

An image is called a portrait in the present invention 
when it contains at least one human face. If an image is 
5 a portrait, a special processing can be performed on this 
image. When determining whether or not an image is a 
portrait, it is necessary to calculate a probability that 
the image is a portrait. 

In the present invention, the probability that an 
10 image is a portrait is called "portrait probability" of 
the image. 

The higher the portrait probability of an image, the 
more likely the image is a portrait. That is an image is 
determined as a portrait if the portrait probability of 
15 the image is high enough. 

Fig. 11 is the flowchart of another method of 
processing an image according to the present invention, 
in which a portrait probability is calculated for the image 
The process begins at step 1101. An image is inputted at 
20 step 1102. 

At step 1103, a plurality of candidates for human face 
regions are identified. And for each of the plurality of 
candidates for human face regions, a probability that the 
candidate for human face region represent a human face is 
25 obtained. Thus, a plurality of probabilities for human 
face regions are obtained. 

Here, the method of obtaining the probabilities for 



39 



Canon Ref: CFP2513US King & Wood Ref: EIE031501US 

these candidates for human face region may be any relevant 
method used in the prior art, or any method described in 
Figs . 2 or 3 . Alternatively, if the image inputted in step 
1102 is an image that has been processed by the method of 
5 the present invention and probabilities for human face 
regions have been saved in the image, the probabilities 
for candidates for human face region can be directly 
retrieved from the image at step 1103 . 

At step 1104, a portrait probability is calculated 
10 for the image based on the plurality of probabilities 
obtained at step 1103 . 

A lot of methods may be used at step 1104 and do not 
constitute any restriction to the method of the present 
invention . 

15 Assume the probabilities for candidates for human 

face region in the image are pi, p2 , p3... and PN. Three 
exemplified methods of calculating the portrait 
probability of an image is given as follows. 

The first exemplified method of calculating the 

20 portrait probability is described below. 

The following description gives three examples of 
calculating the portrait probability based on N 
probabilities for candidates for human face region. Here 
N is an integer greater than or equal to one. 

25 Therefore, the probability that the first candidate 

for human face region is not a human face is 1-pl; the 
probability that none of the first and second candidates 
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for human face region is a human faces is ( 1 -pi ) * ( 1 -p2 ) ; 
and the probability that the image contains no human face 

N 

is (1 - pi) * (1 - p2) * (1 - p3). * * (1 - piV) = ]^[(1 - pi) - 

i=l 

Thus, the probability that the image contains a 
5 candidate for human face region, i.e., the portrait 
probability of the image may be calculated with following 
formula (1) : 

p=i-no- P i). 

i=] 

Where, pi denotes the probability for the i-th 
10 candidate for human face region; N denotes the number of 
candidates for human face region in the image; P denotes 
the portrait probability of the image. 

In the exemplified image 2 00 shown in Fig. 16, there 
are three candidates for human face region (illustrated 
15 in dash line) . The probabilities for the three candidates 
are denoted as Fl, F2 , and F3 . For instance, Fl equals to 
0.8; F2 equals to 0.5 and F3 equals to 0.1. 

If calculated with formula (1) , the portrait 
probability of image 200 in Fig. 16 is 
20 P=l-(1-0.8)*(1-0.5)*(1-0.1)=0.91. 

The second exemplified method of calculating the 
portrait probability is described below. 

First, select the first K greatest probabilities for 
candidates for human face region, which are denoted as psl , 
25 ps2, ps3, ... psK, where K is an integer between 1 and N. 
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The portrait probability of the image is mainly 
dependent on the candidates for human face regions with 
the first K greatest probabilities. Thus, the probability 
that the image contains no human face is 

5 (l-psl)*(\-ps2)*(l-ps3)*..*(l-psK) = O(l-psi) 

i=l 

Thus, the probability that the image contains a 
candidate for human face region, i.e., the portrait 
probability of the image may be calculated with following 
formula (2) : 

10 P=l-fl(l-psi) (2) 

Where, psi denotes the i-th greatest probability for 
candidate for human face region; K denotes the number of 
the selected candidates for human face region in the image; 
• P denotes the portrait probability of the image. 
15 In the image shown in Fig. 16, the first two greatest 

probabilities for candidates for human face region are 0.8, 
0.5. (Here, let K be 2.) 

Thus, if calculated with formula (2) , the portrait 
probability of image 2 00 is 
20 P = l- (l-0.8)*(l-0.5) =0.9. 

The third exemplified method of calculating the 
portrait probability is described below. 

The maximum of the probabilities for candidates for 
human face region is selected as the portrait probability 
25 of the image. That is, the portrait probability of the 
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image is calculated with following formula (3): 
P = Max {pi, p2, p3,... pN} (3) 
Where, pi denotes the probability for the i-th 
candidate for human face region; N denotes the number of 
5 candidates for human face region in the image; P denotes 
the portrait probability of the image. 

In the image shown in Fig. 16, the greatest probability 
for candidate for human face region is 0.8. 

Thus, if calculated with formula (3), the portrait 
10 probability of image 200 is P=0.8. 

At step 1105, it is judged whether the image contains 
human face by comparing the portrait probability with a 
threshold . 

The threshold used here may be in the range of 0.4 
15 to 0.8, and preferably is 0.6. 

At step 1106, the result of the judgment is saved as 
attached information to the image, for example, to the 
header file or footer file of the image in a predetermined 
format . 

20 At step 1107, the portrait probability is saved as 

attached information to the image, for example, to the 
header file or footer file of the image in a predetermined 
format . 

In both of steps 1106 and 1107, the predetermined 
25 format for saving the attached information and additional 
attached information is not important and does not 
constitute a restriction to the present invention. Any 



43 



Canon Ref: CFP2513US King & Wood Ref: EIE031501US 

conventional format or data structure for saving data may 
be used. 

The image to which judging results or probabilities 
have been saved can be used widely. Fig. 12 illustrates 
5 an application of such kind of images. 

The above process ends at step 1108. 

Fig. 12 is the flowchart of another method of 
processing an image to which portrait probability has been 
saved. The process starts at step 1201. At step 1202, an 
10 image to which portrait probability has been saved is 
received . 

As stated above, the probability information might 
be saved as attached information in the header file or 
footer file of the image in a predetermined format. 

15 At step 1203, the portrait probability is retrieved 

from the image, for example from the header file or footer 
file of the image. 

At step 1204, the retrieved probability is compared 
with a threshold. At step 1205, it is decided based on the 

20 comparison result of step 12 04 whether the current image 
is a portrait, i.e., contains human face(s). The above 
threshold can be selected in such a way that any image 
having a portrait probability above the threshold is a 
portrait . 

25 If the judging result of step 1205 is positive, i.e., 

the image is a portrait, the process goes to step 1206; 
otherwise, step 1207. 
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At step 1206, a unique processing method optimized 
for portrait is performed in the image. The unique 
processing method might be performed by a printer, for 
example, a printer driver or an application program for 
5 printer 1713 in Fig. 17, such that human faces will be 
printed with improved printing quality. The unique 
processing method may also be performed by a display, for 
example an application program for display 1714 in Fig. 
17, such that human faces will be displayed with high 
10 quality. 

The unique processing method in step 12 06 may be any 
process performed on an image. For example, it may be a 
process of correcting red eyes in an image, or a process 
of toning down skin color in an image. The value of the 
15 threshold used in step 1204 may be selected as a higher 
value or as a lower value according to different 
applications . 

In the circumstance of red eye correcting, for example, 
it would be unacceptable to users if a portion of an image 

20 (for instance, a red logo) that is not an eye is corrected 
as a red eye. Thus, the threshold in step 1204 could be 
set to a higher value so that less candidates for human 
face regions would be judged as representing human face 
and thus less portions of the image would be corrected as 

25 red eyes . 

In the circumstance of toning down human skin in an 
image, for example, it would be acceptable (sometimes 
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unnoticeable) to users if a portion of an image (such as 
a table) that includes no human skin color is toned down. 
Thus, the threshold in step 1204 could be set to a lower 
value so that more candidates for human face regions would 
5 be judged as representing human face and thus more portions 
of the image would be toned down. 

In one word, the threshold used in step 1204 may be 
selected according to the users' satisfaction of the 
processing result of the image. 
10 At step 1207, a normal processing method is performed 

in the candidate for human face region. 

The process ends at step 1208. 

Fig. 13 is a schematic block diagram of another 
apparatus for processing an image according to the present 

15 invention; 

13 01 denotes an image inputting unit, 13 02 a 
probability for candidate for human face region calculator, 
1303 a portrait probability calculator, 1306 a probability 
writer, 1307 an image outputting unit. The key component 

20 of the apparatus shown in this figure is portrait 
probability calculator 1303. 

Image inputting unit 1301 receives an image and inputs 
its data into the apparatus for processing. Probability 
for candidate for human face region calculator 1302 

25 identifies a plurality of candidates for human face region 
and calculates a probability for each of the identified 
candidate for human face region that this candidate for 
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human face region represents a human face. 

The probabilities for candidates for human face 
region may be calculated by way of the probability 
calculator in Fig. 6, by using the methods shown in Figs. 
5 2 or 3, or any prior methods of calculating probability 
for candidate for human face region. Alternatively, the 
probabilities for candidates for human face region may be 
retrieved from the image if the image has been processed 
by the methods shown in Figs. 2 or 3 and probabilities for 

10 candidates for human face region have been saved in the 
image. Different ways of calculating or obtaining 
probabilities for candidates for human face region do not 
constitute any restriction to the present invention. 

Portrait probability calculator 1303 calculates a 

15 portrait probability based on the probabilities 
calculated or obtained by probability for candidate for 
human face region calculator 1302. The calculation 
performed in portrait probability calculator 1303 may be 
the same as that in step 1104 in Fig. 11. 

20 Probability writer 1306 writes the probability 

calculated by portrait probability calculator 1303 as 
attached information into the image being processed, for 
instance, to its header file or footer file in a 
predetermined format . 

25 The predetermined format for saving the attached 

information and additional attached information is not 
important and does not constitute a restriction to the 
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present invention. Any conventional format or data 
structure for saving data may be used. 

Image outputting unit 13 07 outputs the image for 
further processing . 
5 Fig. 14 is a schematic block diagram of another 

apparatus for processing an image according to the present 
invention . 

1401 denotes an image inputting unit, 1402 a 
probability for candidate for human face region calculator, 
10 1403 a portrait probability calculator. These components 
function in the same way as corresponding components shown 
in Fig. 13. 

1406 denotes a judging unit, 1407 a judging result 
writer, and 1408 an image outputting unit. Judging unit 

15 14 06 judges whether the image is a portrait or not by 
comparing the portrait probability calculated by portrait 
probability calculator 1403 with a threshold. Judging 
result writer 1407 writes the judging result outputted from 
judging unit 14 06 as attached information to the image, 

20 for example, to the header file or footer file of the image 
in a predetermined format . 

The threshold used in judging unit 1407 can be selected 
in the same principle as in step 1204 of Fig. 12. 

The predetermined format for saving the attached 

25 information and additional attached information is not 
important and does not constitute a restriction to the 
present invention. Any conventional format or data 
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structure for saving data may be used. 

Image outputting unit 1408 outputs the image for 
further processing. 

Fig. 15 is a schematic block diagram of another 
5 apparatus for processing an image to which portrait 
probability has been saved. 1501 denotes an image 
inputting unit, 1502 a probability extractor, 1503 a 
judging and control unit, 1504 an image processing unit, 
1505 algorithm for portrait, 1506 algorithm for normal 
10 image, 1507 an image outputting unit. 

Image inputting unit 1501 receives an image and inputs 
its data into the apparatus for processing. Within the 
image, portrait probability for the image has been saved, 
for example, in step 1107 of Fig. 11, and by probability 
15 writer 1306 of Fig. 13. 

Probability extractor 1502 retrieves from the image, 
for example from the header file or footer file of the image, 
the probability for one candidate for human face region. 

Judging and control unit 1503 compares the retrieved 
20 probability with a threshold, decides based on the 
comparison result whether the current image is a portrait 
or not, and controls image processing unit 1504 
accordingly . 

The threshold used in judging and control unit 1503 
25 can be selected in the same principle as in step 1204 of 
Fig. 12. 

Image processing unit 1504 processes the image 
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inputted by image inputting unit 1501 using different 
algorithms such as algorithm for portrait 1505 and 
algorithm for normal image 1506 under the control of 
judging and control unit 1503 . If it is decided by judging 
5 and control unit 1503 that the image is a portrait, image 
processing unit 1504 will process the image with algorithm 
for portrait 1505, otherwise with algorithm for normal 
image 1506. Image processing unit 1504 for instance is the 
component within a printer or a display for processing data 

10 to be printed or displayed. 

Image outputting unit 1507 outputs the image for 
further processing . 

Fig. 17 schematically shows an image processing 
system in which each method shown in Figs. 1 to 5, 11 and 

15 12 can be implemented. The image processing system shown 
in Fig. 17 comprises a CPU (Central Processing Unit) 1701, 
a RAM (Random Access Memory) 1702, a ROM (Read only Memory) 
1703, a system bus 1704, a HD (Hard Disk) controller 1705, 
a keyboard controller 1706, a serial port controller 1707, 

20 a parallel port controller 170 8 , a display controller 170 9 , 
a hard disk 1710 , a keyboard 1711 , a camera 1712, a printer 
1713 and a display 1714. Among these components, connected 
to system bus 1704 are CPU 1701, RAM 1702, ROM 1703, HD 
controller 1705, keyboard controller 1706, serial port 

25 controller 1707, parallel port controller 1708 and display 
controller 1709. Hard disk 1710 is connected to HD 
controller 1705, and keyboard 1711 to keyboard controller 
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1706, camera 1712 to serial port controller 1707, printer 
1713 to parallel port controller 1708, and display 1714 
to display controller 1709. 

The functions of each component in Fig. 17 are well 
5 known in the art and the architecture shown in Fig. 17 is 
conventional. Such an architecture not only applies to 
personal computers, but also applies to hand held devices 
such as Palm PCs, PDAs (personal data assistants) , digital 
cameras, etc. In different applications, some of the 

10 components shown in Fig. 17 may be omitted. For instance, 
if the whole system is a digital camera, parallel port 
controller 1708 and printer 1713 could be omitted, and the 
system can be implemented as a single chip microcomputer. 
If application software is stored in EPROM or other 

15 non-volatile memories, HD controller 1705 and hard disk 
1710 could be omitted. 

The whole system shown in Fig. 17 is controlled by 
computer readable instructions, which are usually stored 
as software in hard disk 1710 (or as stated above, in EPROM, 

20 or other non-volatile memory) . The software can also be 
downloaded from the network (not shown in the figure) . The 
software, either saved in hard disk 1710 or downloaded from 
the network, can be loaded into RAM 1702, and executed by 
CPU 1701 for implementing the functions defined by the 

25 software. 

It involves no inventive work for persons skilled in 
the art to develop one or more pieces of software based 
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on one or more of the flowcharts shown in Figs. 1 to 5, 
11 and 12. The software thus developed will carry out the 
method of training an image processing apparatus as shown 
in Fig. 1, the method of processing an image as shown in 
5 Fig. 2, the method of processing an image as shown in Fig. 
3, the method of processing an image as shown in Fig. 4, 
or the method of identifying a person in an image as shown 
in Fig. 5, the method of processing an image as shown in 
Fig. 11, the method of processing an image as shown in Fig. 
10 12 . 

In some sense, the image processing system shown in 
Fig. 17, if supported by software developed based on 
flowcharts shown in Figs. 1 to 5 , 11 and 12, achieves the 
same functions as the apparatus for processing image shown 
15 in Figs. 6 to 8 and 13 to 15. 

Fig. 18 shows an example of the apparatus for 
processing an image. 

The apparatus shown in Fig. 18 comprises a scanner 
1801, a memory 1802, and a printer 1803. 
20 Scanner 1801 first scans an image and inputs its data 

into memory 1802 for processing. For example, scanner 1801 
scans image 200 in Fig. 16. The portrait probability of 
image 200 for example is 0.91. 

Scanner 1801 is connected to memory 1802. The scanned 
25 image data and the portrait probability of image 200 are 
stored in memory 1802. 

Memory 1802 is connected to printer 1803 . Printer 1803 
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first receives the portrait probability of image 2 00 and 
compares it with a threshold. On the basis of the comparison 
result, printer 1803 decides whether image 200 contains 
a human face. In this example, the portrait probability 
5 of image 200 is 0.91; printer 1803 judges image 200 as a 
portrait . 

Upon the result of the judgment, application software 
program 1803A issues a drawing command group to printer 
driver 1803B of printer 1803. Printer driver 1803B 

10 processes the print request and drawing commands input from 
application software program 1803A, generates print data 
that printer 1803 can print and send the print data to 
printer 1803 . 

Since image 200 is judged as a portrait, printer 1803 

15 will reproduce the human skin. When a printer reproduces 
a human skin, an image with stronger red than an original 
image is apt to be preferable to human sense. 

In the present embodiment, under the control of 
application software program 1803A, the printer makes the 

20 red color, which represent human skin stronger. Thus, 
printer 1803 can reproduce image 200 with a high quality. 

The present invention also provides storage media on 
which codes of computer programs implementing the methods 
of the present invention are recorded. 

25 As a storage medium for supplying a computer program, 

for example, a floppy disk, hard disk, optical disk, 
magneto-optical disk, CD-ROM, CD-R, magnetic tape, 
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nonvolatile memory card, ROM, and the like may be used. 
The functions of the above-mentioned embodiments may 

be implemented not only by executing the readout program 

code by the computer but also by some or all of actual 
5 processing operations executed by an OS (operating system) 

running on the computer on the basis of an instruction of 

the program code . 

As can be seen from the above , the method of the present 

invention provides a fast approach for determining 
10 portrait probability of an image and outputting the 

determined portrait probability for further processing. 

The method allows for the precision determination of a 

portrait under different scales, orientation condition. 

Therefore, in accordance with the present invention, with 
15 the method or apparatus, whether the image is a portrait 

can be quickly and effectively determined, the determined 

image can be further processed. 

The present invention includes a case where, after 

the program codes read from the storage medium are written 
20 in a function expansion card which is inserted into the 

computer or in a memory provided in a function expansion 

unit which is connected to the computer, CPU or the like 

contained in the function expansion card or unit performs 

a part or entire process in accordance with designations 
25 of the program codes and realizes functions of the above 

embodiment . 

In a case where the present invention is applied to 
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the aforesaid storage medium, the storage medium stores 
computer programs for implementing the flowcharts shown 
in Figs. 1 to 5 , 11 and 12. 

The embodiments explained above are specialized to 

5 determine portrait probability of an image and process it, 
however, the present invention is not limited to determine 
portrait probability, it is applicable to other 
determination method, for example, method to detect the 
probability where there is flaw portion on a circuit board. 

10 While the foregoing has been with reference to 

specific embodiments of the invention, it will be 
appreciated by those skilled in the art that these are 
illustrations only and that changes in these embodiments 
can be made without departing from the principles of the 

15 invention, the scope of which is defined by the appended 
claims . 
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